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UNDERSTANDING PANDEMIC LEARNING LOSS 
AND LEARNING RECOVERY: THE ROLE OF 
STUDENT GROWTH & STATEWIDE TESTING 


BACKGROUND 


The COVID-19 pandemic has led to unprecedented changes in the way we live our lives in the United 
States, including the ways in which we educate our students. Beginning in spring 2020, the 
education of students was dramatically interrupted due to the pandemic. Even now in fall-winter 
2020 the majority of students in the United States continue to experience substantial impacts to 
their education. 


Impacts to education and students occur in several ways. The educational impact most often 
discussed is that on student learning: academic impact. However, students’ socio-emotional and 
physical health are also affected. And a downfall in one area can lead to a downfall in another. 
Unfortunately, for the majority of students, the impact in terms of both academic and non-academic 
outcomes has been negative. 


Considering just the negative academic impact, due to 
various, interrelated circumstances such as mode and Because of the pandemic’s 
quality of education services, family resources, and local 
policy toward COVID-19, it is all but certain that the effects 
of the pandemic on the educational lives of students are simply treat everyone to the 
being felt unevenly across the United States. Because of 
the pandemic’'s uneven impact, we cannot simply treat 
everyone with the same intervention and fix what ails ails them academically - there is 
them academically - there is and will be no one-size-fits-all 
vaccine that we administer to each child that cures their 
academic maladies. vaccine that we administer to 


uneven impact, we cannot 
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Indeed, the most challenging aspect of helping students each child that cures all their 
recover academically from the impacts of the pandemic 
will be in answering the question of How to help? 
Interventions will need to be tailored by district, school, 
content area and by student. But it would be a mistake to jump ahead and implement plans and 
interventions of How to help? before first addressing three more basic questions: Who needs 
help?, What do they need help in?, and How much help do they need? 
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Restricting our focus to just the pandemic’s academic impact - often referred to as student learning 
loss or unfinished learning - narrows these three questions down to what must serve as a starting 
point in any effort to overcome the pandemic’s impact on student learning: 


* Who? Who is experiencing or has experienced learning loss? 
- Are students experiencing learning loss related to where they live (region, state, district, or 
school), their demographic subgroup (ethnicity, socio-economic status), their grade, the 
educational services they received (e.g., remote learning) during the pandemic? 
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* What? What have students experienced learning loss in? 
- In what content areas are students experiencing learning loss? Are there major domains or 
sets of standards where students have been affected? 


* How much? How much learning loss have students experienced? 
- What is the magnitude of the learning loss? Is the learning loss larger in one content area 
than in another? Is the learning loss larger for low-achieving students than high-achieving 
students? 


Answering these three questions allows stakeholders to begin to recognize the scope of the impact 
and dedicate attention and resources where they are needed. How well these questions are 
answered affects the quality, granularity and ultimately the success of the interventions. For 
example, consider the most critical of the three questions: How much? An answer to this question 
will help understand the time and effort needed for recovery: Is recovery possible in months or will 
it take years? Coupled with answers to Who? and What? expectations for support and recovery can 
be sensibly calibrated and directed based upon the best evidence available. 


A failure to address these three questions leaves us in the dark amidst a once in a century calamity 
impacting the more than 50 million students in the United States. Such a failure strikes at the heart 
of equity in education efforts that are built upon an accurate assessment of how students, 
particularly disadvantaged students, perform relative to their advantaged peers. But most 
importantly, answering these three questions is only an essential first step that that must be 
followed by more questions supporting the really hard work to determine How to help? especially 
those most in need of assistance. 
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interventions of How to help? before first addressing three more basic 


questions: Who needs help?, What do they need help in?, and How much 


help do they need? 


THE NECESSITY OF DATA 


So how do we answer these questions? An essential prerequisite is data - a substantial amount of 
data on the academic attainment of students. These data come in many forms ranging in granularity 
from teacher observations all the way up to results from large-scale standardized assessments. In 
education, no one, single data point ever tells the whole story. Especially under the current novel 
circumstances, data from many disparate sources are required and need to be comprehensively 
marshalled to answer Who?, What? and How much? 


To that end, over the last two decades in the United States we have developed efficient and effective 
ways to obtain state-wide, high-level data about student achievement through large-scale, state 
summative assessment programs. Billions of dollars have been spent to develop these programs so 
that the tests they comprise can be administered broadly and with fidelity to virtually all students, 
align with state established content standards, and be of high technical quality. Due to the onset of 
the pandemic and associated shutdowns in spring, 2020, statewide testing was cancelled. Barring 
situations that unnecessarily put the health of students, staff and teachers at risk, states should do 
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the best they can to collect these data in 2021 as they will be essential to answer who, what and 
how much. 


Given the rapid onset of the pandemic, any data from a summative assessment carried out in 
spring 2020 would have been of little use in understanding the impact of the pandemic on student 
learning because the disruption to educational programming had just begun at the time of 
proposed administration. Ten months into the pandemic, we are left scrambling for answers to 
Who?, What?, and How much?. Available data from interim assessments administered in fall 2020 
has filled some of the data void. 


Recent learning loss results from Renaissance Learning and NWEA based upon results from interim 
assessment products administered in fall 2020 help to illustrate the importance of having data on 
student attainment and growth and what can be discerned about the impact of the pandemic on 
student learning in the United States.’? Based upon millions of student test results, both studies 
utilized historical growth norms to assess the learning of students between fall 2019 and fall 2020. 
Results from Renaissance and NWEA generally align in terms of both overall learning loss as well as 
differential learning loss between student subgroups: 


* Overall: The pandemic is impacting student learning negatively. Learning loss in mathematics 
is large while that in reading is modest. Learning loss was uneven across grades. 


+ Differential: Learning loss differences between demographic subgroups is minor at this point 
in time and less than feared. 


Assuming the results are consistent with future findings from other data sources (e.g., large scale 
summative assessments), they have significant practical and policy implications. The Renaissance 
STAR student growth data in mathematics, for example, indicated a 15-point drop in median student 
growth percentile (SGP). Such a loss at a state level (let alone at the national level) is unprecedented 
and implies substantial learning loss nationally.? Moreover, it is highly unlikely that a loss of that 
magnitude can be ameliorated based upon a resumption of the normal course of schooling or 
through the implementation of some modest intervention. 


The results, though compelling, have major limitations. Because of the pandemic, the data utilized 
to examine learning loss did not display the same representativeness, particularly with 
disadvantaged students, as is normally the case. If we assume (and this would need to be rigorously 
verified by looking at who tested versus who didn’t) that those who were missing would be 
negatively impacted, on average, as much or more than those who tested, then the results in the 
report present a best-case scenario on student learning loss at this point in time. 


Other limitations include an incomplete understanding on how the allowance for both at-home and 
in-class test administration biases results. Because historical growth norms are derived solely from 
in-class administration, remote administration may yield inaccurate growth results. Because both 
companies can only utilize data based upon their customer base, generalization to locales and 
groups not represented in the data is problematic. And lastly, the interim tests show, at best, 


’ Renaissance Learning (2020). How Kids are Performing: Tracking the impact of COVID-19 on Reading and Mathematics 
Achievement. Special Report Series, Fall 2020 Edition (https://www.renaissance.com/how-kids-are-performing/) 


? Kuhfeld, M, Tarasawa, B., Johnson, A., Ruzek, |, & Lewis K. (2020). Learning during COVID-19: Initial findings on students’ reading 
and math achievement and growth. NWEA. (https://www.nwea.org/content/uploads/2020/1 1/Collaborative-brief-Learning- 
during-COVID-19.NOV2020.pdf) 


3 An SGP difference of -15 corresponds to an effect size of -0.54. 
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modest coverage and alignment with state content : 
standards. Would learning loss look the same, for As we begin 2021, the 

example, as learning loss derived from assessments importance fey i doing our best 
having broad coverage and alignment with state 


standards?4 to administer statewide 


assessments is more important 


These limitations speak to the importance of having 
multiple sources of high-quality assessment data to than ever. Without such data, 
understand and ultimately to help mitigate COVID-19 
learning loss. These data should be derived from 
assessments aligned with accepted academic standards unable to answer Who?, What?, 
that are broadly and appropriately administered. As we 
begin 2021, the importance of doing our best to to 
administer statewide assessments is more important than 
ever. Without such data, we will remain in the dark and unable to answer Who?, What?, and 
How much? As more interim and, presumably, statewide testing data becomes available, sound 
conclusions can be drawn that have the potential to inform significant pedagogical and 

policy related decisions directed to ameliorate the impact of the pandemic on student learning 
going forward. 
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USING STUDENT GROWTH TO ASSESS LEARNING LOSS 


Learning loss. The term rolls easily off the tongue. But what is it really? To explain, consider two 
parallel realities. The first is the pandemic reality in which we currently live. The other is the reality 
where the COVID-19 pandemic never existed. And consider, like in some time-travel movie, being 
able to witness a student living through both of these realities. 


Figures 1 and 2 below illustrate a hypothetical student 


moving through time in each of these realities while we aN al-Molakcimeyina als) eelatel-taalte 
measure their attainment (e.g., in mathematics) at 

different points in time. The result is a timeline of led to a decrease in student 
educational attainment across two realities. Figure 1 Prelsl-Taalte learning (Re growth) 
illustrates the student as their attainment is monitored 

using an interim-like assessment administered three times resulting in a decrease in 

a year. Figure 2 illustrates the students and their Pratlialaallal me aeimelcleaceienla 
attainment using an annual assessment administered in 

the spring. The dots in the figure indicate the level of attainment is defined as 
attainment (vertical axis) for the student at the indicated learning loss. 


point in time (horizontal axis). Red dots/lines are 
associated with our current pandemic reality and black 
dots/lines are associated with the non-pandemic reality. Figure 1 indicates what we currently believe 
is happening to most students based upon Fall 2020 results from interim assessments: The onset of 
the pandemic led to a decrease in student academic growth resulting in a decrease in attainment. 
That decrease in attainment is defined as learning loss. 


*The modest growth declines in reading may be due to content on the interim tests that is less instructionally sensitive and 
aligned with state English language arts content standards than what is generally found on statewide assessments. 
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Figure 1: Visual depiction of learning loss based upon academic attainment measures 
(e.g., interim assessments) given in the fall, winter, and spring of each year. 
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In Figure 2 we observe a student across the two realities but monitor their attainment using an 
annual assessment administered in the spring. Like in Figure 1, black dots are associated with the 
non-pandemic reality and red dots the pandemic reality (our reality). Again, learning loss is the 
decrease in learning between the non-pandemic and pandemic realities. Most important, the figures 
indicate a critical relationship: The decrease in attainment (i.e., learning loss) is actually the outcome 
of the student's decrease in academic growth. 


Figure 2: Visual depiction of learning loss based upon an annual, 
spring summative academic attainment measure. 
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Because we cannot live through both realities and can only observe what is going on in the 
pandemic reality, we must somehow create a basis of comparison that allows us to assess the 
learning loss that has occurred. As we will demonstrate there are sensible approaches to creating 
the basis of comparison that utilize student growth. 


Consider a situation completely unrelated to education. Suppose one was driving somewhere and in 
the midst of the journey a snowstorm slowed progress from 50mph to 20mph. How much “travel 
loss” occurred due to the snowstorm? Like with learning loss depicted in Figure 2 we need a non- 
snowstorm reality as a basis of comparison. The obvious non-snowstorm reality is the one where 
50mph remained the going rate of speed and to use that to determine “travel loss”. 


In extending the travel-during-a-snowstorm example to education-during-a-pandemic, the key is to 
know what constitutes, non-pandemic, academic progress/non-pandemic, academic growth. This 
basis of comparison is a fundamental attribute of the calculation and use of student academic 
growth norms. The most commonly used growth norms, student growth percentiles (SGPs), 
instantiate the entire range of student growth, where 50th percentile growth represents typical 
student growth. Using historically derived growth norms - sometimes referred to as baseline growth 
norms - we can render pandemic student growth relative to non-pandemic growth for all students 
having test scores at time periods such as those illustrated in Figures 1 and 2. 


Whereas under pre-pandemic conditions 50th percentile growth represents typical student progress 
in the same way that 65mph represents a typical driving speed on most US interstate highways, the 
extent to which growth results derived in 2021 are below 50 will demonstrate the extent to which 
the COVID-19 “snowstorm” has slowed student learning. 


This is the approach currently utilized by Renaissance Learning for their interim assessments to 
determine student learning loss. Using historical interim assessment data derived from tests 
administered fall of 2017 and fall of 2018, Renaissance Learning created growth norms showing 
student progress at the 1st through 99th percentiles®. By definition, the median SGP of students for 
the baseline, norming years is 50. This is pre-COVID-19, typical student progress. Applying these 
norms to assessment data from subsequent years indicates the extent to which student growth is 
faster/slower than in the pre-COVID-19 baseline years. As mentioned previously, fall 2019 to fall 
2020 learning loss in mathematics was large (median SGP of 35) whereas learning loss in reading 
was modest (median SGP of 45). 


Due to the lack of spring 2020 summative assessment data, baseline growth norms must be 
adapted to accommodate the missing 2020 data. Anticipating the calculation of student growth 
from 2019 to 2021, it is necessary to create baseline, skip-year growth norms in advance of the 2021 
growth analyses. Doing so utilizes the state’s most recent, historical, skip-year data to calculate 
skip-year growth norms: 2017 to 2019.° Anticipating student growth analyses being conducted in 
spring 2021, baseline, skip-year growth norms have been calculated in more than two dozen states 
for the purpose of understanding 2019 to 2021 student growth relative to historical, non-pandemic 
growth and thus, assess, academic learning loss. 


> The approach to calculating student growth percentiles utilized by Renaissance Learning utilizes prior student attainment 
(up to two prior scaled scores) and current attainment/scaled scores, time elapsed between the current and first prior 
assessments, and testing date of the current assessment as conditioning variables. Details are available in the technical 
manual available here. 


®° We have produced a GitHub repository with step-by-step instructions on how to run historical growth analyses that create 
baseline growth norms (i.e., coefficient matrices) using the R-SGP package, which that can be used by any state or other 
organization to investigate COVID-19 related learning loss. 
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We motivated the discussion in Figures 1 and 2 using an individual student passing through these 
two realities and monitoring their attainment over time. These illustrations could also represent 
learning loss for groups of students where each dot represents the average attainment of students 
in that group. For example, Figure 2 could represent average learning loss for an entire state of 
students in 6th grade mathematics using the annual statewide assessment. In the following we 
consider summary results for large groups to address the overall impact of the pandemic on 
learning loss as well as the eventual recovery. 


IMPACT 


The importance of administering statewide assessments is often premised upon their utility. During 
normal times their utility, in large part, is based upon their use in state accountability systems. Given 
the current pandemic circumstances, it is questionable whether traditional uses within state 
accountability systems are defensible. Currently, determinations of school quality are inextricably 
tangled with pandemic related issues making it impossible to get a clear picture of what exactly a 
good school looks like. Yet beyond traditional accountability determinations, there is great utility in 
administering spring 2021 statewide assessments to inform our approach to recovery. These 
assessments allow us to immediately answer the questions of Who?, What?, and How much? 


Beginning with the question of How much?, Figure 3 illustrates 4 different learning loss scenarios 
based upon results from 2021 statewide assessments for individual or groups of students ranging in 
severity from no impact (upper left) to severe impact (lower right). Given that learning loss will 
impact individual and groups of students unevenly, we expect all of these scenarios to emerge. 
Indeed, with these impact scenarios in mind, determining what students or groups of students are 
associated with each scenario and in what subjects is the critical next step. 


Figure 3: Four potential learning loss scenarios: Clockwise from top-left: 
No learning loss, modest learning loss, moderate learning loss, severe learning loss. 
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The preliminary learning loss data from interim assessments previously discussed suggest that the 
learning loss associated with mathematics is moderate to severe (mean SGP decrease across grades 
of 15) whereas it is modest in reading (mean SGP decrease across grades of 5), But these results 
barely scratch the surface. Statewide assessment data will additionally allow us to answer: 


* Which schools and districts display moderate to severe learning loss? Are there characteristics 
of the schools and districts to explain the learning loss? 


* Which demographic subgroups display moderate to severe learning loss? 


* How severe is learning loss for students with disabilities? How severe is learning loss for 
students identified as gifted and talented? 


* Which schools and districts display modest to no learning loss? How were they able to 
accomplish that? 


* Do results from standards aligned assessments (e.g., state summative assessments) indicate 
the same results? 


* How does learning loss compare between remote and in-person learners? How do other 
COVID-19 related opportunity to learn (OTL) indicators relate to learning loss? 


« And what learning loss is observed across interactions between schools/districts, demographic 
subgroups, and OTL indicators? 


As the latter two questions suggest, a large area of interest is in determining how learning loss 
relates to COVID-19 related groupings like remote learning and even the type of remote learning. 
These data are not routinely collected, especially at the state level. However, if such data can be 
collected with fidelity, then growth results from these groups will be directly and immediately 
applicable to whether such efforts were (un)successful in terms of mitigating learning loss with 
students.’ Baseline-referenced growth norms open up all of these questions to immediate 
investigation by state, district, and/or school personnel. 


One potential area of concern with regard to the calculation of baseline referenced, skip-year 
student growth results in 2021 to assess learning loss is missing data. Participation in spring 
assessments is traditionally quite high (> 95 percent). Hitting this mark in spring 2021 seems 
unlikely. Though the missing data doesn’t impact the previously established skip-year growth norms, 
it would impact the set of students for whom we have growth data in 2021. As of fall 2020, students 
living in rural areas attend schools in-person at much higher rates (63.6%) than students living in 
urban areas (7.8%).° It's conceivable if not likely that spring 2021 assessment coverage might show 
lower participation rates in certain geographic areas and/or with certain student subgroups. 


Missing data will have a larger impact on attainment comparison (percent proficient in 2019 to 2021) 
than to growth and learning loss. Baseline referenced SGPs are uncorrelated with prior attainment 
and therefore less vulnerable to systematic bias due to pandemic related missing data. 
Aggregations to schools with sparse data will encounter the same issue of generalization to the 
entire school as attainment data. However, growth and learning loss inferences to the entire school 


7 See Betebenner, D. & Dadey, N. What’s Even Going On? Collecting Data on Student Experiences to Understand Student Learning in 
light of COVID-19 and Marion, S. Using Opportunity-to-Learn Data to Support Educational Equity. 


8 Gross, B., Opalka, A., and Gundapaneni, P. One step forward, one step back: Public Health Fears Keep America’s School District's 
on a Reopening Treadmill. 
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with sparse growth data can be supported via methods such as propensity score matching on 
school characteristics combined with historical growth data. 


To be clear, these growth data are not meant to support inferences about school quality in the 
sense of traditional accountability. Instead, these data are meant to support an effort - sometimes 
referred to as “small a” accountability - to identify those states, districts, schools, student 
subgroups, and individual students. in need of support that will enable them to recover 
academically from the impacts of the pandemic on their education. 


RECOVERY - =a 
As we transition from the initial stages of the pandemic, As we transition from the initial 
questions about the pandemic's negative impact on the stages of the pandemic, 


academic growth of students will shift to questions about 
what it will it take for students to recover academically? 
Having answers to Who?, What?, and How much? will be negative impact on the 
essential in constructing feasible, appropriately targeted 
and effective recovery plans.’ It is impossible to think 
coherently about recovery when we have no idea how much will shift to questions about 
someone (or some group) needs to recover and in what. 


questions about the pandemic’s 


academic growth of students 


what it will it take for students 
In Figure 4 we extend Figure 2 to include an anticipated 
period of recovery beginning in 2021. This breaks the 
timeline into three phases: Phase 1 - Pre-COVID-19, Phase 
2 - COVID-19 Pandemic, and Phase 3 - COVID-19 Recovery. We borrow some familiar 
conceptualizations of recession shapes to help explain the different ways that academic recovery 
might look. The four scenarios presented are not intended to be exhaustive but provide useful 
archetypes for stakeholders to consider. 


to recover academically? 


The worst case recover scenario depicted in the upper-left panel is no recovery at all: an academic 
downward spiral where the losses a student suffers during the pandemic continue to cascade 
downward. The upper-right panel depicts an L-shaped recovery where the student doesn't fall 
further behind after the initial loss but doesn’t catch up either. The third scenario is the U-shaped 
recovery shown in the lower-left panel. In the U-shaped recovery the student catches up gradually 
over multiple years. The last and best-case scenario, the V-shaped recovery shown in the lower-right 
panel is when the student catches back up immediately. 


° The IES is currently proposing an intense 3-year experiment called Operation Reverse the Loss to investigate interventions to 
help students recover from learning loss. https://ies.ed.gov/director/remarks/10-26-2020.asp 
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Figure 4: Four potential Phase 3 academic recovery scenarios. Clockwise from top-left: No recovery 
(Academic downward spiral/ADS), L-shaped recovery, U-shaped recovery, and V-shaped recovery. 
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It goes without saying that the most desirable recovery for all students is V-shaped. 
Pronouncements that students are “resilient” and will “bounce back” attest to our hopes for that 
V-shaped recovery. Unfortunately, hopes for V-shaped recovery do not compel it to happen. The 
goal should be to maximize the percentage of students having either a V- or U-shaped recoveries 
and minimize the percentage of students who don't recover. Attaining this goal requires an 
understanding of the magnitude of the learning loss being experienced by states, districts, schools 
and other relevant groupings of students. Knowing the magnitude of learning loss will inform just 
how feasible a V-shaped recovery is. 


There is extensive history of state level student growth There is extensive history of 
that can be utilized to inform recovery efforts. Student state level student growth that 
growth percentile analyses of state longitudinal data 
began in 2007. Being norm- and criterion-referenced, the (ot-] as =U al IPA-rom MoM lalie) gan 
results of these analyses cover over 30 states and provide 
a backdrop against which to understand what education 
systems (e.g., States, district, and schools) can sustain in 
terms of academic growth. We know, for example, that typical (i.e., 50th percentile) growth does not 
usually “catch up” non-proficient students to proficiency nor is it sufficient for proficient students to 
reach achievement levels above proficiency. In general, students that start behind tend to stay 
behind and students that start ahead tend to stay ahead." 


recovery efforts. 


© The concept of students needing to “catch-up” is not new in education. Many state accountability systems utilize the growth 
students need to catch-up as a basis of their accountability systems. In some states, grades and content areas, the majority 
of students need to catch-up. Note that with COVID-19, the goal of catching students up to where they would have been 
had the pandemic not occurred is still below where we wanted them to be anyway. 
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Academic recovery from the pandemic is somewhat different than state efforts to catch students up 
to proficiency, but the same logic holds: In our efforts to get students back on track (i.e., the track 
that typical/50th percentile growth produced) we will need student academic growth that exceeds 
what we have historically typically observed. This holds for individuals as well as subgroups of 
students. For example, assume that a state’s mean 2019-2021 SGP derived from summative 
assessments for grade 5 students in mathematics is similar to that found with the Renaissance STAR 
assessment data and equal to 32. To make up that learning loss will require a mean SGP for the 
state of at least 68 in the following year (2021-2022). Such a result is without precedent in the more 
than decade of looking at state level SGPs in the United States. The best we could hope for - and 
what should be planned for - would be a multi-year U-shaped recovery. 


BEWARE THE K-SHAPED RECOVERY 

Which of the Figure 4 scenarios will occur? In reality, all of them will occur. What is unknown and 
what effective policy can influence are the proportions and types of students in each of the 
scenarios. The panels in Figure 5 illustrate this for two types schools: A school serving higher 
achieving students (left panel) and school serving lower achieving students (right panel). During the 
recovery, both of these schools will have individual students experiencing each of the four types of 
recoveries. Some students will see a V-shaped recovery, some an L-shaped recovery, some a 
U-shaped recovery, and some will fall into an academic downward spiral. 


However, the two schools do not experience similar proportions of students in each of the recovery 
scenarios. The weight of each line indicates the proportions of students in that school following that 
recovery path. The two pictures present a situation where students at the higher achieving school 
predominantly follow V- and U-shaped recoveries whereas the students at the lower achieving 
school predominantly follow and L-shaped recovery or fall into an academic death spiral. 


Figure 5: Potential recovery scenarios for two schools: A school serving higher achieving students 
(left) and a school serving lower achieving students (right). The weight of the line indicates the 
proportion of students following that recovery path. 
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Indeed, as Figure 5 illustrates, a strong possibility of utmost concern for our national commitment to 
more equitable outcomes in education is that the most academically dangerous part of the entire 
pandemic for disadvantaged students is not during the pandemic phase but during the recovery 
phase. Without a substantial and committed effort, we could very well see lower achieving, at-risk 
students predominantly falling into the latter two non-recovery scenarios (right panel of Figure 5) 
and advantaged students falling predominantly into the first two (left panel of Figure 5). Called a 
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K-shaped recovery, such a scenario would lead to 
ballooning achievement gaps in the years after the 
pandemic - beyond what they were before the onset of 
the pandemic - due to differential recovery rates. 


Essential in avoiding a K-shaped recovery is to have data 
from statewide assessments that allows us to understand 
the impact (Phase 2) and to monitor the recovery (Phase 3) 
of students. Without Phase 2 data that helps us 
understand Who?, What?, and How much7?, it will be 
impossible to separate impact from recovery. For 
example, without spring 2021 assessment data, widening 
achievement gaps between, for example, ethnic 
subgroups, will likely be attributed to differential impact of 
the pandemic when it is entirely possible that the 
differential recovery is the culprit. 


Beyond monitoring impact and recovery, preventing the 
worst types of recovery scenarios from happening will 
likely require large scale interventions both inside and 
outside the school or classroom. Substantial economic 
and socio-emotional costs due to the pandemic have been 
felt by millions of families. Academic recovery for students 
caught up in these circumstances will require recovering a 
foundation that so many families lost. 


CONCLUSION 


A strong possibility of utmost 


concern for those committed to 
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education is that the most 
academically dangerous part of 
the entire pandemic for 
disadvantaged students is not 
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during the recovery phase. 


Called a K-shaped recovery, 
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ballooning achievement gaps in 


the years after the pandemic 


- beyond what they were before 
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to differential recovery rates. 


As we live through the COVID-19 pandemic, foresee its end, and consider the policies and 
interventions necessary to cope with and ultimately recover from the learning losses students 
experience, we must commit ourselves to the collection of high-quality data to help inform 
education policy related to the COVID-19 pandemic. This includes data derived from large-scale, 
statewide, summative assessments. Data derived from large-scale, statewide assessments have 
become fundamental to our understanding of student educational attainment and growth in the 
United States over the last two decades. Despite their traditional use in state accountability systems 
being highly problematic for the 2020-2021 school year, these data are essential in answering the 
questions of Who?, What?, and How much? to inform academic recovery from the pandemic. 


This is not to suggest that we test at all costs and or recklessly risk the health of students and 
teachers. It is to suggest that we test as best we can and marshal the well-established statewide 
data collection efforts to help us understand and ultimately mitigate the impact of the pandemic 


on students. 


The prescriptions outlined in this paper are not novel or new. In any given year there are millions of 
students who experience a dip in their academic attainment (i.e., experience low growth). The 
COVID-19 pandemic has just substantially increased that number. Historically, state accountability 
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systems built using statewide assessment results have Sealers 2021 statewide 
been inadequate in identifying these students falling back. 


Past inadequacy is no excuse for not redoubling efforts assessment data will be an 
now to leverage statewide assessment results to help indispensable eMule helping 
students recover academically. 

to illuminate the circumstances 
As of January 2021, the United States is almost completely , ; 
in the dark as to pandemic’s impact on student learning. we find ourselves in, to chart a 
Early interim assessment results have given us a glimpse ey-1damcel aN lac me-laremRomantelaline)s 
of the magnitude of the impact but have been extremely ; 
limited otherwise. We need much more and much better our journey along that path. 


data to intelligently move forward. 


Spring 2021 statewide assessment data will be an indispensable flashlight, helping to illuminate the 
circumstances we find ourselves in, to chart a path forward, and to monitor our journey along that 
path. This will be most critical for traditionally disadvantaged groups. Absent this data, efforts to 
address inequity in our education system will likely take a significant step backward as the 
disadvantaged suffer the worst consequences of the pandemic while the advantaged predominantly 
recover. Preventing this outcome will likely require an unprecedented effort nationwide built upon 
high quality academic attainment and growth data like that provided by statewide assessments. 
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